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Abstract 

An expanded vision of evaluation is offered that 
accommodates the use of both descriptive information and 
evaluative ^ information, and which connects evaluation in the 
naturalistic paradigm with evaluation in the rationalistic 
paradigm for making informed decisions within programs of 
education and development. The model of "Information Development 
and Evaluation" as presented herein is the conceptual structure 
around which vibrant cultures of information can be built within 
program ^ organizations; and such organizations can then socialize 
reflective practitioners, promote- collaborative group decisions 
and action, and shape themselves into learning organizations. 



Prepared for the Regional Training Workshop on Monitoring 
and Evaluation of Nonformal Basic Education, Chiang Rai, 
Thailand, 7-25 November, 1994. 



INFORMED DECISIONS IN A CULTURE OF INFORMATION: 
UPDATING A MODEL OF INFORMATION DEVELOPMENT AND EVALUATION 



Introduction 

^ Informed decisions are the crux of the matter in all 
purposive action from policy development, planning, project 
design, to management, implementation and evaluation. To ensure 
regularity of informed decisions by individual agents within 
organizations, individual decision-makers must be supported by 
vibrant "cultures of information" i.e., organizational 
cultures that both value and use information continuously and 
systematically collected and retrieved for use by all. 

Evaluation theory is, of course, about making informed 
decisions, but by its relatively greater focus on " evaluative 
information" it seems to have neglected the equally, if not the 
more important, role of " descriptive information" in decision 
making. Needed is a model that encompasses an expanded view of 
evaluation, accommodating both descriptive and evaluative 
information -- both of which are equally necessary in decision 
making [1] . Such a model should also serve as a structure around 
which a culture of information can emerge within a program 
organization, in turn, enabling reflective practice, 
collaborative action and learning organizations. 

The Multiple Contexts that Shaped the Model 

The triad of ideology, theory, and practical experience 
together have attended the development and updating of the model 
now presented below: 

The Model in the Context of Practice 

The model presented herein was born in the context of 
practice, in a series of evaluation workshops for middle level 
workers in literacy programs in some Eastern, Central and 
Southern African countries from 1976 to 1987 (Bhola, 1989b) ; and 
was tested-in-use in the course of evaluation field studies 
conducted in Botswana, Malawi and Zimbabwe (Bhola, 1988, 1989a, 
1990a) . 

During these many encounters with practitioners in the 
field, it was found that practitioners wanted, first and 
foremost , to be informed about what was going on on the around 
within the framework of their programs. The questions, "How 
well?" and "To what affect?" would come later. It was learned 
that most decisions made by program administrators in on-going 
programs were related to the needs for program expansion, teacher 
recruitment and choice and provision of reading materials. What 
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these literacy practitioners and administrators needed first for 
making all these decisions was descriptive information based on 
data which should typically be found in a Management Information 
System (MIS) , howsoever rudimentary such and MIS was to be. Most 
of the information that practitioners within literacy programs 
wanted did not seem to require sophisticated evaluation studies. 
Literacy workers were already committed to the idea that some 
literacy was better than no literacy. They were not ready to 
commit scarce resources either for "questioning" their own 
commitment to the program, or for answering fine-tuned, 
"scientific" questions in relation to correlations and 
comparisons among and between organizational patterns, 
methodological approaches and reading materials written in 
"different instructional styles. 

That is not to say that data about affects are not 
required at all. Surely, data on literacy affects were needed. 
What the decision makers did not need in the beginning were 
rationalistic evaluations establishing causes and correlations 
between factors and effects. What they did need was personal 
testimony from new literates, community leaders and other 
extension workers in the field to the effect that the programs 
they had launched were useful. They wanted to know what the 
programs were doing to these people, at home and at work. Such 
data could be supplied only by evaluations conducted in the 
naturalistic mode. But since such information was not being 
produced, most of these decision makers ended up using the weaker 
impressionistic and anecdotal data. 

This experience from the field spread over many years 
pointed out to the necessity of developing an evaluation model 
that would be based in the real-world needs of the practitioners. 
It was thus the product of a practical necessity. 

The Theoretical Context of the Model : 
Within the Paradigm Debate 

There is, of course, always a dialectical relationship 
between theory and practice. The initial conceptualization of 
the model and its later elaboration had been possible because of 
the very nature and quality of the paradigm debate which at the 
time was beginning to move towards a paradigm dialog (Guba 1990) . 
Many evaluators were heeding Firestone (1990) who had suggested 
that paradigms were not systems of rules connected in a network 
of deductive operations. Rather, paradigms were cultures of 
research which as such can be brought into dialectical 
relationships between seemingly opposite views. One could indeed 
take the position, I should add, that statistical interpretation 
of reality as part of the positivist paradigm could be considered 
to be one "special construction" of reality. In and of itself, 
this special construction may be a partial description of reality 
but it is yet a "reality" that does hold within some particular 
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contexts and on the basis of which some decision makers would be 
ready to act . 

In the language of Cronbach (1980) , part of a program 
context could be defined as the "context of control" and thereby 
connected to positivist assumptions. Another part of a program 
context could be defined as the "context of accommodation" and 
may be have to make constructionist assumptions. The challenge 
was to make appropriate assumptions in appropriate contexts, and 
avoid being mired in mixed assumptions. Thus, in the same one 
program, all of the questions such as the following are relevant 
and necessary: How many students are there and what is their age 
structure? How are literacy and numeracy correlated? How do 
people, right or wrong, feel about their personal objectives 
being fulfilled by the program? How are lives of the new 
literate being changed through their acquisition and use of 
literacy skills? What new symbioses are appearing between 
orality and literacy. Different paradigms and different modes of 
information development and evaluation can, of course, be used as 
appropriate to answer each different question. 

Descriptive and evaluative information --a false 
dichotomy? It was also becoming clear that description and 
evaluation was a false dichotomy. As Donald Davidson (1968) 
suggested descriptive information by being situated in wider 
social and linguistic context and through "redescriptions" can 
become evaluative information. Thus, it is proper to work with 
the larger category of information and then think cf information 
as being of two kinds descriptive information and evaluative 
information . 

The Ideological Stance of the Model 

The ideological spirit of the age was also congenial to 
the model as it was being elaboration. The ideology of 
development now asked for endogenous planning and innovation, 
collaborative decision making, and participative evaluation. The 
model responded to these ideological positions in affirming 
internal evaluation and promoting reintegration of evaluation 
back into the day to day practice of education and development. 
It did seem that the over-prof essionalization of evaluation as a 
specialization had blown asunder the necessary symbiotic 
relationship between purposive action and evaluating [2] . 

While the level of prof essionalization of evaluation over 
the last thirty years has been impressive and while some 
evaluation studies may have influenced national and international 
policy debate, there have been some unanticipated negative 
consequences. One unfortunate consequence of this otherwise 
impressive prof essionalization of evaluation may have been this 
disassociation of evaluation from implementation. The 
practitioner who implements purposive action does not feel 
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obliged to evaluate his or her own practice. This is reinforced 
by the professional evaluator who sees the practitioner neither 
as competent nor objective in the evaluation role. Thus the 
practitioner undergoes a split in the middle of the 
practitioner's role, and is seriously disabled in regard to 
reflecting on his or her own practice and from taking informed 
decisions . 

Concomitantly, all internal evaluation when undertaken is 
discounted as inferior and undependable . Thereby, the 
practitioner loses professional control of his or her work to the 
outsider. It is the outsider who then reconstructs hierarchies 
of significant objectives, establishes the criteria of success, 
selects sources of data that would be preferred, and methods of 
measurement and then makes pronouncements on the success or 
failure or programs. The situation is ironic. The outsiders 
first use local people and insiders to translate and administer 
instruments initially written in a metropolitan language. Then 
the outsiders use the insiders to translate back into the 
metropolitan language the interview or questionnaire data 
collected in an indigenous language since the so-called external 
and objective evaluators neither have the time, nor the language, 
nor social access to the subjects of the study. 

In the case of bilateral and multilateral projects, the 
control we have talked about may often shift to foreign agencies 
and institutions thousands of miles away. The consequences of 
such shift of control can be serious and need not be recounted 
here in any detail. We accept, of course, that donor nations, 
within the context of technical assistance, are entitled to know 
how the resources made available by them are being spent and with 
what effectiveness. However, evaluation patterns should not be 
imposed from the outside to distort local objectives and methods 
and to destroy local initiative and responsiveness to popular 
needs . 

The model offered below takes the ideological stance that 
without undoing the professionalization of evaluation, internal 
evaluation should be given a place of its own in the process of 
generating information for informed decisions. In rehabilitatinq 
internal evaluation, the model points to the necessity of 
rejoining "valuing" with "doing", that is, implementing with 
evaluating. The practitioner should not merely be allowed nut 
expected to be engaged in continuous self -monitoring 
description and evaluation of practice in using feedback for 
feedforward. The amputation of the practitioner's role must be 
avoided. We should all be practitioners and evaluators at the 
same one time . 

Ideology, theory and experiential realities have thus 
converged in the model presented below. 
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A Description of the Model 
of Information Development and Evaluation ; 
An Expanded View of Evaluation 

An expanded view of evaluation is claimed for this model 
(see figure 1 attached), first, because the typically used 
category of "evaluation" is expanded into the larger category of 
"information" and thereby descriptive information is added to 
evaluative information in our conception of the processes of 
making informed decisions; second, because, both paradigms of 
making warranted assertions about reality naturalistic- 
constructivist and rationalist-positivist have been included; 
and third, because evaluators are invited to rise above and go 
beyond occasional, untimely, stand-alone, discrete and sometimes 
disruptive evaluation studies; and instead, be challenged to 
create vibrant "cultures of information" within programs and 
organizations (Bhola 1991a, 1991d) . 

While we argue for the development of overall cultures of 
information within program, the trianguJ.ated approach to 
information development and evaluati' - offered in the model is 
applicable to individual evaluation , idies as well that may be 
conducted as part of the overall program evaluation and in the 
process of moving towards a vibrant culture of information. (See 
figure 2 attached) . 

A Description of the Model 

As can be seen, three modes of information gathering are 
presented Management Information System (MIS), Naturalistic 
Evaluation (NE) and Rationalistic Evaluation (RE) . There is a 
triangular relationship among them. The box including the 
"Management Information System (MIS) juts into the upper half of 
the rectangle to indicate that the two categories of descriptive 
information and evaluative information are not completely 
separate and inclusive but rather overlapping. The so-called 
descriptive information has evaluative content in that good 
descriptions implicate decision makers toward particular 
prescriptions . 



[INSERT HERE THE FIGURE: An Integrated Model of 
"Information Development and Evaluation."] 



[INSERT HERE THE FIGURE: Integration of Methodological 
Approaches within a Stand-alone Information 
Development / Evaluation Study.] 
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The Management Information System (MIS) 



It should be noted that the MIS is the foundational 
component of the model and should be the "first" to be installed 
and implemented within a program system. The MIS will include 
descriptive information about the size and scope of a program. 
An MIS in its initial state need not be complete and 
comprehensive covering the total set of all possible indicators 
of a program, but should expand with the growing information 
needs. MIS's need not always be computerized, and could easily 
be a paper-and-pencil affairs. Ideally such information will be 
generated in the very process of the implementation of a program. 

To each MIS, its own SIM the necessity of a 
corresponding set of integrated materials (SIM) . Every MIS must 
be complemented with a good SIM (Set of Integrated Materials) in 
the form of text, voice and image. Instructional materials, 
periodical reports, newspaper stories, pictures and photographs, 
specimens of writing, tapes, videos, should all be stored in this 
SIM. Each item should be indexed in relation to the numerical 
data included in the MIS. 

Qualitative data, once quantified, may not remain 
qualitative . Evaluators may sometime claim to have used 
qualitative approaches in their evaluations when in reality they 
may not done so. What they may have done is to have obtained 
some open ended responses from their respondents and then scored 
them to covert those responses into numerical data. This 
quantitative transformation of open ended data does not qualify 
as qualitative approach. 

Adding a true qualitative component to the MIS . A 
question is often asked: Is it possible to add qualitative data 
to an MIS? The answer is "Yes!" Modern computers today can 
store immense amounts of data and provide instant random access 
to various sets of data within a large data set. Such data sets 
need not all be numerical, but can also include narratives, that 
is, computers can include contextual, and qualitative information 
for interpretive use. Figure 3 attached to the paper indicates 
how this might be possible to do. Relevant computer software may 
be available or available computer software may be easily 
adaptable to this use. 



[PLACE HERE FIGURE: Showing the Structural Design 
for a Computer Program Linking "Numerical Data" 
with "Narrative for Interpretation. ] 
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With the use of a format as proposed, numerical data and 
narrative for interpretation on an individual, a particular 
group, a community or a region can be linked. At the same time 
inter-individual, inter-group, inter -district , and inter-regional 
comparisons can be made. 

The narrative for interpretation will, of course, have to 
be developed through analysis of contextual materials such as 
historical data, policy and planning documents, and by conducting 
naturalistic studies focussed on individual, groups, communities, 
and regions that are included in the numerical data sets in the 
computer-based MIS . ~~ ~ ~~~~ 



Naturalistic Evaluation (NE) 

An MIS-SIM while a necessary component of the information 
development and evaluation plan is not by itself sufficient. 
Naturalistic Evaluation (NE) is an absolutely necessary 
supplement and complement to the MIS, Naturalistic evaluation 
gets the "second" priority in this model of information 
development. There is a two way relationship between MIS and NE 
-- MIS will provide "empirical" grounding to many of the 
assertions made on the basis of NE, while NE will make MIS data 
meaningful, explaining how people within programs of literacy, 
for instance, were experiencing those programs and how they were 
reconstructing their lives at home and work as new literates. 

Let us be reminded that the inclusion of qualitative data 
called the "narrative for interpretation" to be stored in the MIS 
will almost all be generated through naturalistic evaluation. 
Some additional materials will come from historical documents, 
five year plans and other policy materials. 

Rationalistic Evaluation (RE) 

Rationalistic evaluation gets the last position in this 
model of information development and evaluation. The last can 
sometimes mean the lease, but it certainly does not mean complete 
exclusion. RE studies will be required at some times in the life 
of a program to develop information dealing with comparisons 
between various pedagogical or organizational approaches to 
literacy work and there may be occasions when correlations 
between various outputs have to be studied. To answer these 
types of questions, we will have to use RE studies. What we are 
saying is that in the early life of a program, RE type studies 
are seldom needed, MIS and NE studies are much more in demand. 
Quite often an RE study would be based on samples from data 
already in the MIS, Sometimes additional data may have to be 
collected. Quite often, an RE study on completion will be 
incorporated within an extended MIS. 
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Towards a Vibrant Culture of Information 



Students of human cultures have often talked of how 
structure and culture exist in a relationship of mutual shaping. 
To help promote a viable and vibrant culture of information, an 
intelligent use of an appropriate structure (in this case the 
integrated model of information development and evaluation) will 
have to be made. 

A culture of information can be conceptualized as an 
emergence from: (1) the values and norms held by participants in 
a systemin relation to the development, validation, and 
utilization of information in decision making; (2) the patterns 
of social and organizational relationships within which 
information is held, withheld, and used; and (3) the tools used 
for collecting, collating, and storing information. 

An Ideal Culture of Informatio n 

All human organizations have communication patterns and, 
therefore, have a culture of information. But these cultures of 
information may not be healthy. In an ideal culture of 
information, information is valued, without discounting 
intuition; disinformation is not deliberately allowed to enter or 
to emanate from within the program system; information is 
legitimized and validated using non-political criteria and within 
collaborative-participative patterns; information is obtained 
from all possible categories of stakeholders within the program 
system as well from the community outside; and information 
developed or obtained is allowed to flow freely both vertically 
and horizontally. 

In healthy cultures of information, information collection 
is not separated from information utilization. For example, 
information developed by a functionary is used by that 
functionary and others at his level without having to wait for 
orders from above. In other words, every member of the 
organization has the right to use the available information in 
making decisions within his or her jurisdiction at his or her 
level. ^ Finally, tools of information collection, and methods of 
collation, storing and retrieval of information are appropriate. 

Creating an Ideal Culture of Information 

Creating cultures of information is a slow, complex and 
difficult task, involving changes in or renewal of existing norms 
and values about validating, legitimizing and using information; 
in ways and patterns of holding, withholding and utilizing 
information; and in the tools and techniques currently in use in 
gathering, collating and storing information. This is 
contradictory process that involves both "strengthening and 
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subverting, reinforcing and renewing" the culture which now 
exists and is undergoing renewal (Bhola 1990c) . 

Cultures of information can not be created in a social and 
political vacuum. We need to understand that the culture of the 
surrounding organization, of the community, and the politics of 
the society in which programs are being implemented will together 
determine the limits and the possibilities of such cultures being 
recreated. 

In the context of an international project involving 
intercultural comparisons (Belanger 1991, Ouane 1991), several 
interlocking processes will be involved: intercultural 
communication; changes in the authority structure of the 
organization and in relationships between the program 
organization and the community; resocialization of both 
functionaries within the organization and of those they serve 
within the community; and training of those within and outside 
the program organization. 

Let us now change the perspective and look ahead from the 
vantage point of teams of practitioners involved in developing 
cultures of information within projects. 

Renewing Cultures of Information form the Inside 

Renewing and recreating cultures from the inside is not 
merely a matter of issuing new office orders. It is unlearning 
some of the old values and internalizing some new ones; it is 
learning new ways of relating, and organizing; and it also 
involves learning new technologies through a long collaborative 
and participative process. 

There are two parts to the process: 

A. Making the existing culture of information "visible" . 
Make the existing culture of information visible and identify its 
inadequacies, distortions and dysfunctions (in relation to an 
ideal culture of information) ; and 

B. Renewing the existing culture of information to make it 
more "viable" a nd "vibrant" . Make changes so that the new 
culture-in-the-making becomes more viable and vibrant (desirable 
in relation to the criteria of an ideal culture of information 
and feasible in relation to the realities of the context of the 
program) . 

A . P ractical steps in making the existing culture of 
information visible 

1. A vanguard group in a continuous seminar . Establish a 
vanguard group, with functionaries from all levels of the 
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organization to meet in the context of a continuous seminar so as 
to keep on learning about a particular educational program in the 
country; and to acquire appropriate skills in making the current 
culture of information in your country project, first, visible 
and then renewing the existing culture of information to make it 
more viable and vibrant. 

2. Taking stock of tools of information collection . In 
making the existing culture of information visible begin by 
examining the existing tools of information collection. Collect 
and make a list of all the tools of information collection, 
collation, record and storage of information now in use within 
the program. In each case ask the following questions: When, and 
why this tool was introduced, by whom and who designed this tool? 
Has the tool been revised, when and why? Has the tool worked 
well in terms of the expectations from it at the time it was 
introduced? What information has been developed from the use of 
the tool and to what use has this information been put? Taken 
together do the tools of information gathering currently in use 
fulfil the information needs cf the program in question? and What 
other information do decision makers seem to need to increase the 
effectiveness of the program? 

3 . Information salience, selection, flow, and control . 
Begin by asking the question: Is the need for dependable 
information salient within the program organization? Do formal 
information-handling roles exist within the organization? Are 
appropriate resources committed to information development and 
utilization? Examine the patterns of information selection, and 
information flow within the program system and to the 
stakeholders outside, and identify the patterns of control of 
information by various functionaries at" various levels. 

Some further questions such as the following may be 
raised: Is information defined only as formally collected 
information or does it also include "informally" obtained 
information? if "informal" information is part of the 
information pool, how is it validated, and made part of the 
usable information within the program? Who has the right and the 
opportunity to contribute information to the program's pool of 
information? Doss it include the beneficiaries of the program in 
question? In addition to formal evaluations, what approaches are 
used to generate quantitative and qualitative information? What 
use is made, for example, of reports written by officials after 
field visits, of communications from the field initiated by field 
workers, or by community leaders, etc.? Once information has 
become formalized as information about the program, how does this 
information flow within the system horizontally and 
vertically? Does all information become semi-confidential or 
fully confidential once it has been processed? Is too much of 
information unnecessarily considered confidential? Is 
information flow controlled by level of hierarchy? What kind of 

11 



ERLC 



12 



information can functionaries use, as they find it, without 
having to get "approval" from above? 

4. The values ana norm's of information use . In a more 
crude form, this is the issue of the politics of information. 
The following questions should be asked: What are the current 
values about informed choice? Is information collection a mere 
ritual? Is the morality and the economics of non-use of 
information understood? Is there democratic access to 
information? Is there misuse of information by omission or 
commission? 

B . Practical steps in the process of renewal of the 
existing culture of information viable 

All the questions raised in the section A above will not 
only make the culture of information visible, it will also 
provide ideas and data on how to make it more viable and vibrant. 
Whereas Section A above described the analytical phase, the 
present Section B is focussed on the design phase. The design 
phase will involve the creation of a structure (a body of tools, 
rules and work patterns) that provides the technology; and a 
super-structure (of values and norms) that will provide the soul 
for the new culture of information. 



The Dividends of a Culture of Information : 
Connections with Theories of Reflective Practice, 
Collaborative Action, and Learning Organizations 

Reflective practice, collaborative inquiry and action, and 
the learning organization are three of the most important ana 
recurrent themes of social and behavioral sciences today. All 
these three traditions are post-positivist , humanistic and 
participative. The triumvirate is thus paradigmatically and 
theoretically sound and ideologically congenial. The democratic 
ethos of our times encourages creativity, effectiveness and 
fulfillment as it asks professionals to engage in reflective 
practice. Collaborative inquiry and action is again in the 
democratic spirit. Finally, the learning organization by 
definition asks organizational actors, irrespective of location 
in the hierarchy in the organization, to learn and to inter- 
learn, and in the process claim ownership of the organization's 
mission and operations. What is ideologically good is also 
theoretically sound. Reflective practice does work better. 
Collaborative groups are more productive and creative. Learning 
organizations treat the insiders humanely and compete with the 
outsider successfully. 

It should now be stated that the relationship between the 
quality of the culture of information of an organization and th^ 
social and behavioral life of that organization are two aspects 
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of the same reality. Information is the glue that connects roles 
and rules and tasks within an organization together into a living 
system. The availability, accessibility, and patterns of 
utilization of information within an organization shape not only 
the effectiveness of decision-making but also the commitments and 
competencies of role performers and the overall health of the 
organization itself. The culture of information is the 
environment within which reflective practice, collaborative 
inquiry and action will germinate and an organization will become 
a learning organization. Stated in another way, an organization 
without a viable, vibrant and robust culture of information is 
unlikely to accommodate reflective practice on the part of its 
members, will inhibit collaborative inquiry and action, and will 
not become a learning organization (see figure 4) . 



[PLACE HERE THE FIGURE: A Model of the Process of 
Emergence of a Viable Culture of Information 
Within Learning Organization.] 



Reflective practice and action at the individual level . 
Reflective practice is a particular epistemology of practice that 
goes beyond technical rationality to ref lection-in-action . But a 
reflective dialog between the teacher and the taught while 
intuitive and creative, is neither mysterious nor uninformed. In 
fact it is the status of being informed that permits the 
reflective thinker to make intuitive leaps. Reflection does not 
take place in an information vacuum. To be in a culture of 
information is to know-in-action. 

Collaborative learning and action in groups . We are 
seeing a rejection of the norms of rugged individualism, selfish 
self-sufficiency and competition. We are discovering the uses of 
cooperation and collaboration. It is asserted that intellect 
flourishes in social settings of interaction and exchange, and 
that knowledge produced in social settings through interaction 
has greater validity. Similarly collaborative action has greater 
salience and possibility of implementation. 

Once, again, it is clear that good participation and 
collaboration can not reside in mere sharing of ignorance, 
enveloped in the indulgent emotion of being all in togetherness. 
Nor is cooperation and collaboration a pseudo democracy of 
unexamined choices. Good collaborative thinking and action lives 
on a healthy diet of information. Many collaborative strategies 
are per se strategies of information development and information 
use . 

The learning organization: the institutional level . The 
concept of the learning organization is a continuation of the 
theoretical traditions of the healthy, democratic, and humanistic 
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organization. Theory development in the area of learning 
organizations has come from Shrivastva (1983) , Pucik (1988) , 
Argyris (1990), Senge (1990), March (1991) and Simon (1991). 
The essence of the theory of the learning organization is that 
(i) an aggregate of learning individuals do not simply add up to 
a learning organization though continuing self -learning and 
training of members of the organization is an essential condition 
of a learning organization; and (ii) that being a learning 
organization is dependent on storing and structuring 
organizational experience in both the quantitative and 
qualitative modes, having open ended processes for making 
decisions , self-conscious , collaborative planning, and engaging 
in systemic thinking to be able to connect with the living 
environments in the world around. In other words, a learning 
organization must be a culture of information as well. 

Back to Cultures of Organizations 

Organizational learning, on the other hand, is premised on 
systemic thinking. Peter Senge (1990) says that when teams think 
systemically, they develop a language that enables them to 
discuss complex issues. As they move beyond "mere openness" to 
"reflective openness" they move from simple discussion of values 
and ideas to a willingness to look inward and challenge their own 
thinking, values and ideas. Obviously, these processes can be 
actualized by the well-informed, living and working in an 
environment of openness. 

Sarason (1991) most directly raises the important question 
"How do you build into a new school self -correcting forums and 
mechanisms so absent in our schools? (p. 120) . He then goes on 
to suggest that "The task is not evaluation in the narrow sense, 
but development of an organizational culture that makes self- 
correction a norm and not a war (p. 120) ." The statements are 
easily translatable in the language of open communication and a 
culture of information as we have discussed in the above . 

To sum, organizations that subscribe to Theory Y, that are 
healthy organizations and are learning organizations are also 
"cultures of information." Communication and control are a part 
of the definition of all social systems. Consequently, any human 
system (from family, community, organization, to the state) , by 
the very fact of being in existence, will have a culture of 
information embodied in it. Therefore, we do not create cultures 
of information from scratch within human systems. Cultures of 
information good or bad, pre-exist. We only assist in the 
process of making those existing cultures of information more 
sufficient, more functional and more dynamic. 

Connections with Planning, implementation and evaluation . 
As the graphic presentation of the connections among and between 
the expanded vision of evaluation, the culture of information, 
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reflective practice, collaborative learning and action, and 
learning organizations shows, such a culture will have 
consequences in all the various aspects of purposive action from 
planning through implementation to evaluation. 



Conclusion 

Does the model as elaborated even have a chance of 
succeeding? Is it not an exercise in Utopian imagination? Are 
evaluation professionals going to like this suggested move toward 
recombining "evaluating" with "implementing" and the accompanying 
shift back to internal evaluation? Are the pundits of 
methodology going to accept this paradigmatic hybrid? Are the 
practitioners themselves going to like this "additional work" of 
evaluating their own programs and reflecting on them becoming 
uncomfortable as professionals and vulnerable as officials? Are 
the higher-up within organizational hierarchies going to like the 
openness that the model as a structure for a culture proposes 
after all information is power, and power is exercised through 
being secretive about information and by manipulative of 
communication? Would the power-holders not simply damn the 
culture of ^ information and capture the structure of information, 
to centralize information and power for their own good? 

The answers to all these questions may not be comforting. 
But then that would be the situation in relation to most of these 
questions in any setting of social change that questions 
complacency, demands commitments, requires higher competency, and 
disturbs vested interests rooted in the status quo. We have to 
assume a moral order, and hope that we have people in leadership 
who have good intentions. We must take up the challenge of 
approximating to the ideal, howsoever hard and slow the road 
ahead. 
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NOTES 



1. The model now presented is an update of the model 
developed within a series of evaluation workshops in East, 
Central and Southern African countries during 1976-87 and which 
was first published in H.S. Bhola, Evaluating Literacy for 
Development; Projects, Programs and Campaigns (Hamburg: Unesco 
Institute for Education / Bonn: German Foundation for 
International Development, 1989/90) . [The book has since become 
available also in French, Spanish, Arabic and Farsi.] 

2. An idea of the level of prof essionalization of evaluation 
can be obtained from The Preparation of Professional Evaluators : 
Issues, Perspectives, and Programs (James W. Altschuld, Molly 
Engle, eds . ) . Special issue of New Directions for Program 
Evaluation . A Publication of the American Evaluation 
Association, No. 62, Summer 1994. San Francisco, CA: Jossey- 
Bass Publishers. The editors and authors, all professional 
evaluators do not regret this prof essionalization. They do, 
however, point to several issues and contradictions. 
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Figure: Integration of methodological approaches 
within a stand-alone information 
development / evaluation study. 

©Bhola (1993) 



ERIC 



25 



FIGURE 3 




D is the Program District 
R is the Program Region 
larger than the District 



Figure: Showing the structural design for a computer program linking 
"numerical data" and the corresponding "narrative for 
interpretation" for various units of analysis, e.g., individuals, 
groups, program districts and the next larger program units 
(Bhola, 1994). & 
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